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Abstract. It is still an open question whether a compact embedded hypersur- 
face in the Euclidean space 1R"+-'^ with constant mean curvature and spherical 
boundary is necessarily a hyperplanar ball or a spherical cap, even in the sim- 
plest case of surfaces in R^. In a recent paper the first and third authors 
have shown that this is true for the case of hypersurfaces in R"'+^ with constant 
scalar curvature, and more generally, hypersurfaces with constant higher order 
r-mean curvature, when r > 2. In this paper we deal with some aspects of the 
classical problem above, by considering it in a more general context. Specifi- 
cally, our starting general ambient space is an orientable Riemannian manifold 
M , where we will consider a general geometric configuration consisting of an 
immersed hypersurface into M with boundary on an oriented hypersurface P 
of M . For such a geometric configuration, we study the relationship between 
the geometry of the hypersurface along its boundary and the geometry of its 
boundary as a hypersurface of P, as well as the geometry of P as a hypersur- 
face of M. Our approach allows us to derive, among others, interesting results 
for the case where the ambient space has constant curvature (the Euclidean 
space R"'+^, the hyperbolic space H"+^, and the sphere S"+^). In particular, 
we are able to extend the symmetry results given in |3] to the case of hyper- 
surfaces with constant higher order r-mean curvature in the hyperbolic space 
and in the sphere. 



1. Introduction 

An old problem in classical differential geometry consists on finding all compact 
surfaces in Euclidean space with constant mean curvature and circular boundary. 
As is well known, a circle C in is the boundary of two spherical caps with 
constant mean curvature H for any positive number H, less than or equal to the 
inverse of the radius of the circle C. A natural question to ask ^UJ is whether 
a compact constant mean curvature surface in 'E? which is bounded by a circle 
is necessarily a spherical cap or a flat disc. Actually, a constant mean curvature 
surface with circular boundary is the mathematical model of a soap bubble which 
has its boundary on a round hoop, and the surfaces we almost always observe are 
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spherical caps, so that it is natural to ask if these are the only solutions. In [T^ 
Kapouleas gave a negative answer to this question by showing that there exist 
examples of higher genus compact, non-spherical immersed surfaces with constant 
mean curvature in R"^ bounded by a circle. However, it has been conjectured that 
there must be a positive answer to this question if one requires in addition that the 
surface has genus zero or that it is embedded 9' . 

In recent years, several authors have obtained some partial answers to these 
problems. For instance, Barbosa 0] |3] proved that the only compact immersed 
surfaces with constant mean curvature H ^ and circular boundary which are 
contained either in a sphere or in a cilinder of radius l/|i?| are the spherical caps. 
On the other hand, in the genus zero case the first author, jointly with Lopez 
and Palmer, has showed that the only stable constant mean curvature immersed 
surfaces of disc type which are bounded by a circle are spherical caps (see also 
[5] for another characterization of spherical caps as the only stable examples, given 
by Barbosa and Jorge under a stronger idea of stability). 

It is clear that this classical question can be stated in a more general context 
as follows. Let S"^^ be a compact (n — l)-dimensional submanifold contained in 
a hyperplane 11 C R""*"^, and let M" be an n-dimensional connected orientable 
manifold with smooth boundary dM. As usual, M is said to be a hypersurface of 
M"+i with boundary S if there exists an immersion ip : M" such that the 

immersion ip restricted to the boundary dM is a diffeomorphism onto S. In this 
context, the classical question above consists on finding the compact hypersurfaces 
in R"+^ with constant mean curvature whose boundary E is a round (n— l)-sphere. 
At this point, it is interesting to recall that a classical result by Alexandrov [J states 
that round spheres are the only closed hypersurfaces with constant mean curvature 
which are embedded in Euclidean space R"+^ (here by closed we mean compact 
and without boundary). More recently, Alexandrov theorem was extended by Ros 
to the case of constant scalar curvature 22 , and more generally to the case of 
hypersurfaces with constant higher order mean curvature |28j , showing that round 
spheres are the only closed embedded hypersurfaces with constant r-mean curvature 
in M"+i. 

As for the case of non-empty boundary, in |13| Koiso gave a new interpretation 
of the problem by studying under what conditions the symmetries of the boundary 
S C n of a non-zero constant mean curvature hypersurface M in M""*"^ are inher- 
ited by the whole hypersurface. She showed that this necessarily occurs when the 
hypersurface M is embedded and it does not intersect the outside of S in 11; as a 
consequence, if the boundary E is a round (n — l)-sphere, then M is symmetric 
with respect to every hyperplane through the center of S which is orthogonal to 
n, and hence M must be a spherical cap. Related to Koiso's symmetry theorem, 
Brito, Sa Earp, Meeks and Rosenberg ^ also showed that when E is strictly convex 
and M is embedded and transverse to 11 along the boundary dM, then M is en- 
tirely contained in one of the half-spaces of M""'"-'^ determined by 11 and, therefore, 
the so called Alexandrov reflection technique 1 implies that M inherites all the 
symmetries of S. In particular, if E is a round sphere, then AI must be a spherical 
cap. Here, transversality means that the hypersurface M is never tangent to the 
hyperplane H along its boundary. In what follows, we will use the term symmetry 
result to refer to a result of this type. 
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The technique introduced in |H] makes an extensive use of two essential ingredi- 
ents, the abovementioned Alexandrov reflection technique, and an integral formula 
first found by Kusner |15| , which is now known as the flux formula. This fact in- 
dicates that the symmetry result in [3] can be extended from two new viewpoints: 
by considering constant mean curvature hypersurfaces in other space forms; or by 
considering the case of hypersurfaces with constant higher order r-mean curvature. 
From the first point of view, Nelli and Rosenberg studied the case of hypersur- 
faces in hyperbolic space IE1I"+^, and, more recently. Lira ^7] considered the case 
of hypersurfaces in the sphere establishing corresponding symmetry results 

for the case of constant mean curvature. On the other hand, in 24 Rosenberg es- 
tablished a version of the flux formula for hypersurfaces with constant higher order 
r-mean curvature in Euclidean space R"+^, and applied it to extend the symmetry 
result given in ^ to the case of the higher order r-mean curvatures. 

In this paper, we will deal with some aspects of the classical problem above. 
Our initial strategy is to study this problem in a more general context. Specifically, 
our general ambient space will be an (n+ l)-dimensional connected orientable Rie- 
mannian manifold M, where we will consider the following geometric configuration 
(for the details, see Section 01). Let us fix P" C M an orientable connected totally 
geodesic hypersurface in M, and let E"^^ C P be an orientable {n — l)-dimensional 
compact embedded submanifold contained in P". Consider A/" an n-dimensional 
connected orientable manifold with smooth boundary dM. Then, M is said to be a 
hypersurface of M with boundary E if there exists an immersion ip : M" — + m"'~^ 
such that the immersion tfj restricted to the boundary dM is a diffeomorphism onto 
S. From this geometric configuration, the following question, closely related to the 
symmetry problem, naturally arises: 

How is the geometry of M along its boundary dM related to the geometry of the 
inclusion S C P? 

A first partial answer to this question is given by the following expression, which 
holds along the boundary dM and for every 1 < r < n — 1 (see Corollary |OJ , 

Here stands for the r-th classical Newton transformation associated to the second 
fundamental form on M (see Section O for the details), is the outward pointing 
unit conormal vector field along dM, £, is the unitary normal field of P C M, and 
Sr — Sr(Ti, . . . , r„_i) is the r-th elementary symmetric function of ti, . . . , r,i_i, the 
principal curvatures of E C P with respect to the outward pointing unitary normal. 
As a first consequence of this expression, we obtain a very strong relationship 
between the transversality of M with respect to P along the boundary dM, and 
the ellipticity on M of the r-th Newtom tranformation T^, that is, the positivity 
of the quadratic form associated to T^. This fact, along with Theorem 7.3 in 
|24j . allows us to state the following symmetry theorem for hypersurfaces in K"+^ 
(Theorem EHJ: 

Let E be an strictly convex compact {n — 1)- dimensional submanifold in a hyper- 
plane n C R""*"^, and let tp : M" M"+^ be a compact embedded hypersurface with 
boundary E. Let us assume that for a given 2 < r < n, the r-mean curvature Hr 
of M is a nonzero constant . Then M has all the symmetries ofE. In particular, 
if the boundary E is a round {n — l)-sphere o/R"+^, then M is a spherical cap. 
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As a consequence, we can conclude that the conjecture of the spherical cap [H] 
is true for the case of embedded hypersurfaces with constant r-mean curvature 
in M"+^, when r > 2. This includes, in particular, the case of constant scalar 
curvature, when r = 2 

In order to extend this symmetry result to the case of hypersurfaces in hyperbolic 
space and hypersurfaces in the sphere, it is necessary to establish a certain flux 
formula, which is one of the key ingredients of the used techniques. For that 
reason, Section |S1 is devoted to derive a general flux formula for the considered 
geometric configuration in the case where the Riemannian ambient space M is 
equipped with a conformal vector field (ProDOsition l8.l|l . Our general flux formula 
becomes specially simple when the ambient space has constant sectional curvature, 
and the conformal vector field is indeed a Killing vector field. In that case, we are 
able to extend the flux formula given by Rosenberg in |24l Theorem 7.2] to the case 
of the other space forms, as follows fCorollarv l8.2|) : 

n+l 

Let if) ■ M" M he an immersed compact orientable hypersurface with bound- 
ary dM , and let _D" be a compact orientable hypersurface with boundary dD = dM . 
Assume that M U D is an oriented n-cycle of M , and let N and nn be the unit 
normal fields which orient M and D, respectively. Assume that M has constant 
sectional curvature. If the r-mean curvature Hr is constant, 1 < r < n, then for 
every Killing vector field Y G X(M) the following flux formula holds 



where v is the outward pointing conormal to M along dM . 

As first applications of our general flux formula, we derive some interesting esti- 
mates for the volume of minimal hypersurfaces with boundary on a geodesic sphere 
of the ambient space, in the case where the ambient space is the Euclidean space 
fCorollarv l8.4|l . the hyperbolic space fCorollarv l8.5() . or the sphere fCorollarv l8.6|l . 

On the other hand, and as another application of our flux formula and the ex- 
pression for {TrV,v) given in Corollarv l6.ll fsee above), we establish in Section 
some interesting estimates for the constant r-mean curvature in terms of the geom- 
etry of the boundary. Specifically, when the ambient space is the Euclidean space 
we obtain the following (Theorem |^^: 

Let S be an orientable {n — 1)- dimensional compact submanifold in a hyperplane 
P C M"^"'^, and let %Ij : A/" M"+-'^ be an orientable immersed compact (connected) 
hypersurface with boundary E — ^j{dM) and constant r-mean curvature Hr, 1 < 
r < n. Then 



where hr^i stands for the (r — l)-mean curvature ofEdP, and D is the domain 
in P hounded by S. In particular, when Y, is a round [n — l)-sphere of radius g it 
follows that 



This estimate is the natural generalization of an estimate first obtained by Bar- 
bosa in the case of constant mean curvature (r = 1) On the other hand, when 
the ambient space is the hyperbolic space, our estimate reads as follows (Theo- 
rem I^J: 





0<\Hr\ < -. 
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Let E be an orientahle {n — 1)- dimensional compact submanifold contained in 
a totally geodesic hyperplane P C H"+"'^, and let ip : Af" H"+"'^ be an orientable 
immersed compact connected hypersurface with boundary S = ip{dM) and constant 
r-mean curvature Hr, 1 < r < n. Then 



Here stands for the (r — l)-mean curvature of Yi G P, D is the domain in P 

bounded by S, and C = maxs coshp > 1, where g{p) is the geodesic distance along 
P between a fixed arbitrary point a G int(£') and p. In particular, when T, is a 
geodesic sphere in P of geodesic radius g, it follows that 



Similarly, for the case of hypersurfaces in the sphere, our estimate states the 
following f Theorem 19. 

Let S be an orientable (n — 1)- dimensional compact submanifold contained in 
an open totally geodesic hemisphere P+ C S"+^, and let ip : M" — > S""'""'^ be an 
orientable immersed compact connected hypersurface with boundary S = ip{dM) 
and constant r-mean curvature Hr, 1 < r < n. Then 



Here /i^-i stands for the (r — l)-mean curvature of Yi <Z P , D is the domain in P^ 
bounded by S, and C = maxs cos g/ min/^ cos g, where g{p) is the geodesic distance 
along P+ between a fixed arbitrary point a € int(Z3) and p. In particular, when E 
is a geodesic sphere in P^ of geodesic radius g < 7r/2, it follows that 



Finally, the two remaining sections of the paper are devoted to the extension 
of our symmetry results to the case of hypersurfaces in the hyperbolic space and 
hypersurfaces in the sphere. Specifically, in Section IIUI we obtain the following 
symmetry result for hypersurfaces in hyperbolic space fTheorem llU.lf) : 

Let S"~^ be an strictly convex compact (n— 1)- dimensional (connected) subman- 
ifold of a totally geodesic hyperplane P" C H""*"^, and let M" C H""*"^ be a compact 
(connected) embedded hypersurface with boundary S. Let us assume that for a given 
2 < r < n, the r-mean curvature Hj. of M is a nonzero constant. Then M has all 
the symmetries of Y. In particular, when the boundary Y is a geodesic sphere in 
P" C H"^-'^, then M is a spherical cap. 

As a consequence, we can conclude, as in the Euclidean case, that the conjecture 
of the spherical cap is true for the case of embedded hypersurfaces with constant 
r-mean curvature in hyperbolic space, when r > 2. Finally, in the case of hyper- 
surfaces in the sphere, we state the following symmetry result (Theorem II 

Let S"^^ be a convex {n — \)- dimensional submanifold of a totally geodesic n- 
sphere P" C S"^^, and let M" C S""*"^ be a compact (connected) embedded hyper- 
surface with boundary S. Let us assume that M is contained in an open hemisphere 
S"+S and that the r-mean curvature Hr of M is a nonzero constant, for a given 
2 < r < n. Then M has all the symmetries ofS. In particular, when the boundary 
Y is a geodesic sphere in P" C §"+^, then M is a spherical cap. 




0<\Hr\< coth>. 




0<\Hr\< cof g. 
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In particular, the only compact embedded hypersurfaces in §" with constant 
r-mean curvature Hr ^ (with 2 < r <.n) and spherical boundary arc the spherical 
caps. 

2. PRELIMINARIES 
n+1 

Throughout this paper, M will denote an (n+ l)-dimensional connected ori- 
entable Riemannian manifold, and ( , ) and V will stand for its Riemannian metric 
and its Levi-Civita connection, respectively. Let M" be an n-dimensional connected 
orientable manifold with smooth boundary dM] M is said to be a hypersurface of 
M if there exists an isometric immersion -0 : M" m"^ . In that case, since M 
and M are both orientable, we may choose along ip{M) a globally defined unit 
normal vector field N, and we may assume that M is oriented by N. If V denotes 
the Levi-Civita connection on M then the Gauss and Weingarten formulae for the 
immersion are given, respectively by 

(2.1) VvW ^VvW + {AV,W)N, 
and 

(2.2) A{V) = -VyN, 

for all tangent vector fields V,W £ X{M). 

Here A : X{M) — > X{M) defines the shape operator (or the second fundamental 
form) of the hypersurface with respect to N. The curvature tensor R of the hyper- 
surface M is described in terms of A and the curvature tensor R of the ambient 
space M by the so called Gauss equation, which can be written as 

(2.3) R{U, V)W ^ (R{U, V)W)'^ + {AU, W)AV - {AV, W)AU 

for all tangent vector fields U,V,W £ X{M), where ^ denotes projection on X{M). 
Observe that our criterion here for the definition of the curvature tensor is the one 
in |2ni- On the other hand, the Codazzi equation of the hypersurface describes the 
normal component of R{U,V)W in terms of the derivative of the shape operator, 
and it is given by 

(2.4) {R{U, V)W, N) = {{VvA)U - {WuA)V, W) 

where Vc/v4 denotes the covariant derivative of A. In particular, when the ambient 
space has constant sectional curvature, then R{U,V)W is tangent to M for every 
U,V,W e XiM), and ^ becomes 

(2.5) {VvA)U = iVuA)V. 

As is well known, A is a self-adjoint linear operator in each tangent plane TpM , 
and its eigenvalues Ki(p), . . . , Knip) are the principal curvatures of the hypersurface. 
Associated to the shape operator there are n algebraic invariants given by 

Sr{p) = ar{Kiip), . . . , Kn{p)), 1 < r < n. 

where ct,. : K." ^ M is the elementary symmetric functions in K" given by 



fJ^(xi, . . . , Xn) / ^ • . ■ Xi, 
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Observe that the characteristic polynomial of A can be writen in terms of the 5r's 
as 

n 

(2.6) det{tl - A) = ^ (-!)'■ S-^r-^ 

The r-mean curvature Hr of the hypersurface is then defined by 

In particular, when r — 1 then Hi = (l/n)trace(A) — H is the mean curvature of 
M, which is the main extrinsic curvature of the hypersurface. On the other hand, 
when r — 2, H2 defines a geometric quantity which is related to the (intrinsic) 
scalar curvature of the hypersurface. Indeed, it follows from the Gauss equation 
(|2.3(l that the Ricci curvature of M is given by 

Ric(;7, V) = mc{U, V) - {R{U, N)V, N) + nH{AU, V) - {AU, AV), 

for U,V E X{M), where Ric stands for the Ricci curvature of the ambient space 
M. Therefore, the scalar curvature S of the hypersurface M is 

S = trace(Ric) = 5^-2 Ric(N, N) + n{n - 1)^2. 

For instance, if the ambient space has constant sectional curvature c then 

(2.7) S = n{n-l)(c + H2). 

3. The Newton transformations 

The classical Newton transformations • X{M) X{M) are defined induc- 
tively from A by 

To = I and Tr = Sri — ATr-i, ^ 1^ r < n, 
where / denotes the identity in X{M), or equivalently by 

Tr = Sri - Sr-lA +■■■ + {-ly-^SiA'-^ + (-l)M''. 

Note that by the Cayley-Hamilton theorem, we have T„ — 0. 

Let us recall that each Tr is also a self-adjoint linear operator in each tangent 
plane TpM which commutes with A. Indeed, A and Tr can be simultaneously diag- 
onalized; if {ei, . . . , e„} are the eigenvectors of A corresponding to the eigenvalues 
Ki(p), . . . , Kn{p), respectively, then they are also the eigenvectors of Tr correspond- 
ing to the eigenvalues of Tr, and Tr{ei) = fJ-i.rip)ei with 

Mi,r(p) = ^^^(ki(p),...,k„(p)) = V K,^{p)---K,^{p), 

ll<---<lr 

for every \ < i < n. From here it can be easily seen that 

(3.1) trace(Tr) = {n — r)Sr ~ CrHr, 

(3.2) trace(ATr) = (r + l)Sr+i ^ CrHr+i, 

where Cr = {n — r-)(") = (r + 1)(^";^). For the details, we refer the reader to the 
classical paper by Reilly (see also [21] for a more accesible modern treatment 
by Rosenberg). 
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On the other hand, the divergence of Tr is defined by 

n 

divM % = trace(VT,) = ^(Ve,T,)(e,), 

where {ei, . . . ,e„} is a local orthonormal frame on M. Below we will compute 
divM Tr, which will be necessary for its later use. 

Lemma 3.1. The divergence of the Newton transformations Tr are given by the 
following inductive formula, 

(3.3) divAf To = 

n 

divM Tr - -A(divM Tr-i) ~ ^ (i?(N, T,„ieOe.)^ , 

i=l 

where R stands for the curvature tensor of M , and (i?(N, y)VF)^ denotes the 
tangential component of R{J>i,V)W. Equivalently, for every tangent field V e 
X{M) it follows 

r n 

(3.4) (divM Tr,V) = (i?(N,T,_,e,)e„ AJ-iy). 

3=1 »=1 

The expression (13 .41) has been recently obtained also by Lima in using a 
very different argument to ours. 

Proof. It is clear that div^f To = div^f / — 0. When r > 1, from the inductive 
definition of Tr we have for V,W € X{M) 

{\7vTr)W = {\7Sr,V)W -\7v{ATr^i)W 

= {VSr, V)W - {VvA)iTr-lW) - A{{VvTr-l)W), 

so that 

n n 
diVM Tr = ^(Ve.T,)(e,) = ^Sr " ^ ( Ve, A) (T,_ 1 6,) - ^(diVM Tr-l). 
2=1 i=l 

Using now the Codazzi equation (|2.4(l we get for V G X{M) 

{{\/,^A)(Tr^ie,),V) = {{\7e,A)V,Tr-ie,) 

= ((VyA)e„r,_ie,} + (:R(F,e,)T,_ie„N) 
= (T,,_i((VvA)e,),e,) + (i?(N, T,_ie,)e„ F). 

Therefore, 

(3.5) (divMT^,V) ^ (V5,r, V) -trace(r,._iVyA) 



^(i?(N,T,^ieOe„y) - (A(divM Tr-i),V). 



1=1 



Using now equation (4.4) in f2J we have that 

trace(T^_iVyA) = (V5^,y}, 

which jointly with H3.5|l gives l|3.3|l . Finally, equation H3.4II follows easily from H3.3(l 
by an inductive argument. □ 
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In particular, when the ambient Riemannian space M has constant sectional 
curvature, then (i?(N, y)T/F)^ = for every tangent vector fields V,W X{AI) 
and equation (|3.4|) implies that divjvf = for every r. 

Corollary 3.2. When the ambient Riemannian space M has constant sectional 
curvature, then the Newton transformations are divergence-free: divM — for 
each r. 

4. A GEOMETRIC CONFIGURATION 

Throughout this paper, we will be particularly interested in the following geo- 
metric configuration, which is suggested by the classical question stated in the 
Introduction. Let P" C M be an orientable connected hypersurface in M, and 
let S"^^ c P be an orientable {n — l)-dimcnsional compact embedded subman- 

ifold contained in P". Let -0 : M" Ad be an orientable compact connected 
hypersurface in M with smooth boundary dAI. As usual, Af is said to be a hy- 
persurface with boundary S if the immersion tp restricted to the boundary dM is a 
diffeomorphism onto E. The following question naturally arises from this geometric 
configuration: 

How is the geometry of M along its boundary dM related to the 
geometry of the inclusion S C P and the inclusion P <Z M? 

In what follows, we will study this question. Let us start by choosing the ori- 
entation of this configuration. Let us consider the hypersurface M oriented by a 
globally defined unit normal vector field N. The orientation of M induces a natural 
orientation on its boundary as follows: given a point p e dM , a basis {ui, . . . , Vn-i\ 
for Tp{dM) is said to be positively oriented if {u, ui, . . . , f„_i} is a positively ori- 
ented basis for TpM, whenever u G TpM is outward pointing. We will denote by 
v the outward pointing unit conormal vector field along dM. By means of the 
diffeomorphism ip\oM '■ dM E, the orientation of dM is induced on each con- 
nected component of S. On each connected component Pq of P, we distinguish 
a connected component Eq C Po of E. Let rjo be the unitary vector field normal 
to Eo in Pq which points outward with respect to the domain in Pq bounded by 
Eq. Now, we choose the unique unitary vector field normal to Pq in M which 
is compatible with rjo and with the orientation of Eq. We note that the chosen 
orientation of Pq given by the field determines a unique choice to the unitary 
vector field 77 normal to each components of E in Pq such that 7/|so — rjo- We repeat 
this process to the others connected components of P and hence we obtain unitary 
vector fields rj normal to E in P, and ^ normal to P in M . With this choice, given 
a point p £ E, a basis {vi, . . . , for TpE is positively oriented if and only if 

{r]{p), vi, . . . , Vn-i} is a positively oriented basis for TpP. 

Let {ei, . . . ,e„_i} be a (locally defined) positively oriented frame field along a 
fixed connected component of dM. Using this frame, we can write v — ei x . . . x 
e„_i X N, and similarly 7y = ei x . . . x e„_i x ^, since det(i>, ei, . . . , e„_i, N) = 1 = 
det(?7, ei, . . . , e„_i, ^). From these expressions we easily compute 

77 = ei X . . . X e„„i X ^ = ei x . . . x e„_i X ((^,N)N+ 

= (e,N);.-(e,i^)N, 

that is, 

(4.1) (ry,i/} = (e,N} and (7?, N) = -(C, i^). 
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Let As (respectively, Ap) denote the shape operator of E" ^ C P" (respectively, 

n-j-l 

P" C M ) with respect to the unit normal vector field 77 (respectively, £,). It 
then follows that 



n-l 

E 



for every 1 < i, j < n ~ 1, and also 



ri-l 



so that from (|4.1() we have that 

(4.2) (Ae„e,} = -(Ase„ej)(C,J^) + (Ape„ej)(e,N}. 

Equality H4.2|l above shows us that it is not possible to go further without any 
additional geometric hypothesis on the geometry of the inclusion P C M. A hypoth- 
esis of relevant geometric nature, and which is also technically quite appropriate 
for us, consists on assuming the umbilicity of P C M. Then, from now on let us 
suppose that P is a totally umbilical hypersurface in M. Therefore, there exists a 
smooth function A £ C°°{P) such that Ap = XI, where / denotes the identity in 
X{P), and 14.2|l becomes 

(4.3) (Ae„e,) = -(Ase„ej}(C,J^) +A(e,N}%, l<^,J<r^-l. 

We now suppose that the basis {ei, . . . , e„_i} C Tp{dM) on the boundary is cho- 
sen such that it is formed by eigenvectors of Ay: , and let us denote its corresponding 
eigenvalues by Ti{p), . . . , T„_i(p). In other words, 

A^ei = Tid, 1 < i < n - 1. 

Hence by (I4.3|l . (Ae^, Cj) = when i ^ j, and for each p e dM , the matrix of A in 
the orthonormal basis {ei, . . . , e„_i, v} of TpM is given by 

{Av,ei) \ 



( 



(4.4) 



A 



7i 










72 



(Ai/, 62) 



7«-i 
{Av, e„_i) 



{Av, e„_i) 
{Av,v) ) 





V {Av,ex) {Av,e2) 

where 7^ = -Ti(^, i/) 4- A(^, N) for 1 < i < ?i - 1. 

Now we compute the characteristic polynomial of A. To do that, we begin by 
observing that 

{t -7„-i)det (</„_! 



(4.5) 



where 



det(i/„ - A) 



A(7i 



,7«-2j 



(Ai^, e„_i) (i-7i) 



71 
72 



- A(7i, . . . ,7„-2)) 

{t - 7„„2) 





V (Az/,ei) {Av,ei) 



ln-2 

{Au, e„_ 



{Av,ei) \ 
{Av, 62) 
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Therefore, applying a simple induction argument on n in H4.5|l . we obtain that the 
characteristic polynomial of A is given by 

ri-l 
i=0 

n-1 n-2 
i=l j=0 

where 3^(7) (respectively 5^(7^)) stands for the elementary symmetric functions of 
71, . . . ,7„-i, (respectively 71, . . . ,7^, . . . ,7„_i), and, as usual, so(7) = ■so(7i) = 1 
by definition. Comparing the terms of above polynomials, we conclude from (|2.6() 
that the symmetric function of curvature Sr of the hypersurface M, at a boundary 
point p G dAI, is given by 

(4.6) Si = sii-f) + {Ai^,iy), 

n-1 

(4.7) ^2 = s2h) + si{j){A,.,,.)-Y,{Ai^.e,)\ 

i=l 
n-1 

(4.8) Sr = Sr{j) + Sr-l{'j){Al^,l') ~'^Sr-2{7i){^'^,eif: 

4=1 

for 3 < r < n. 

5. The Newton transformations on the boundary 

Observe that expressions 1)4.6(1 (|4.7|l and (|4.8|) provide us with a partial answer 
to our initial question, since it relates the geometry of the hypersurface M along 
its boundary dM (given by the r-curvature Sr) to the geometry of S C P and the 
geometry of P C M (given by Sr(7)). But this expression it is not still satisfactory 
for our purposes. We need the following essential auxiliary result. 

Lemma 5.1. Let P" C M be an orientable totally umbilical hypersurface in M 
and let Yi <Z P be an orientable (n — 1)- dimensional compact submanifold in P" . 

Let : M" M ^ be an orientable connected hypersurface with boundary S = 
'ip{dM), and let v stands for the outward pointing unit conormal vector field along 
dM C M . Then, along the boundary dM and for every 1 < r < n — 1, it holds 

(5.1) {Trl^,iy) ^ Sr(j) = Srijl, ■ ■ ■ ,7«-l), 

where "fi = — Ti(^, v) + A(^, N) for 1 < i < n—l. Here ri, . . . , t„_i are the principal 
curvatures ofEcP with respect to the outward pointing unitary normal, N is the 
unitary normal field of M , ^ is the unitary normal field of P d M , and A is the 
umbilicity factor of P d M (with respect to S^). 

Proof. We will use induction on r. First, observe that from 14.6|l it follows that 
115.11) holds for r = 1. For a given 2 < r < n — 1, suppose that 

(5.2) {Tji,,iy)^s,ij) 
holds for all 1 < j < 7' — 1. Observe that 

n-1 

Av {Av, ei)ei + {Av, i')v, 

i=l 



12 



LUIS J. ALfAS, JORGE H.S. DE LIRA, AND J. MIGUEL MALACARNE 



SO that from the inductive definition of Tr and (|5.2|1 we conclude that 

(5.3) {TrV,v) = Sr — {Tr-lV, Av) 

n-1 

= Sr - {Tr-iv,iy){Aiy,iy) - ^ (T,._iJ/, (Aj/, e^) 

i=l 
n-1 

i=i 

On the other hand, we also know from (|4.4|l that 

Ac, = -iic, + {Aiy,ei)iy, 
so that from our induction hypothesis (|5.2I) we have for every 1 < j < r — 1, 

{TjU,e.i) = -{Tj^iiy,Aei) = ~ji{Tj^iiy, a) ~ Sj^i{j){Aiy,ei). 
This implies by a recursive argument that 

r-2 

(5.4) {Tr-iJ^,e,) = -{Aiy,e,)Y,i~'^y Sr-2~jhH = -{Aiy,e,)sr-2{^), 

3=0 

since it is not difficult to see that 

ni 

Smili) = ^(-l)^Sm-j(7)7^ 

for every 1 < m < n — 1 . Using now H5.4|l in (|5.3|) , along with H4.8|l , we conclude 
that 

n-l 

{TrV, v) ^ Sr~ Sr-l{-l){Av, v) + Sr-2{li){Av , tif = Sr{l). 

i=l 

This finishes the proof of Lemma [5. II □ 

Now, it remains to know how the elementary symmetric function 3, (7) can be 
expressed in terms of the principal curvatures ti, . . . , t„_i of the inclusion S C P 
and the umbilicity factor A of P C M . To see this, let us write 7i = ai + (3, where 
ai — --Ti{^, v) and /3 = A(^, N), for each i = 1, . . . , n — 1. 

Lemma 5.2. 

^'•(7) = E (" ^ ^\f3^~HM^ 1 < r < n - 1. 
j=o V ^ / 

Proof. Recall that Sr(7) can be defined by the following polynomial identity H2.6|) . 

Tl-l 

^(-l)'-s.(7)r-i-- = it~j,)...{t- 7„_i). 

r=0 

Since each 7^ = a; + the right hand side of this equality can be written as follows 

ri-l 

{{t - /?) - ai) . . . {{t a„_i) = ^ (-!)■'" (a) - /3)"-i--'". 
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On the other hand, computing the right hand side of this last equahty, we obtain 

1 n—l—j 



7i — J. IL — ± /L — i-—J / 1 '\ 

— n T — n I — n ^ / 



n—l—k—j 



j=0 j=0 k=0 

which after a re-ordering becomes 



n-l 



n—l—r 



r=0 \j=0 

Therefore, we have obtained the following equality between polynomials 
n-l n-l / / — 1 — X \ 

r=0 r=0 \j=0 V J / J 

which concludes the proof. □ 

We summarize what was made above as follows. 

Proposition 5.3. Let P" C M be an orientable totally umbilical hypersurface in 
M and lefEcP be an orientable (n— 1)- dimensional compact submanifold in P". 
Let %l) : Af" — > M be an orientable hypersurface with boundary S = %l>{dM), and 
let V stands for the outward pointing unit conormal vector field along dM C M . 
Then, along the boundary dM and for every 1 < r < n — 1, it holds that 

(5.5) (T,^, z.) = ^(-1)^- ("^ 7 ^) X^-^{C, Ny-^{C, vYs,. 

3=0 \ ^ J / 

Here Sj = Sj(ri, . . . , r„_i), < j < n— 1, are the elementary symmetric functions of 
Ti, . . . , T„_i, the principal curvatures ofEcP with respect to the outward pointing 
unitary normal, N is the unitary normal field of M , ^ is the unitary normal field 
of P C M, and A is the umbilicity factor of P C M (with respect to 



6. TRANSVERSALITY VERSUS Ellipticity 

The relationship between the S'^'s and the Sr(7)'s given in 14.6|l . (|4.7I) and H4.8|l . 
as well as the expression for {TrV, v) given in (|5.5|) becomes specially simple in the 
case where the inclusion P C M is totally geodesic, that is, when A = 0. In that 
case 7i = — Ti(^, v), and we have the following. 

Corollary 6.1. Let E be an orientable (n—1)- dimensional compact submanifold in 

n+l n+1 

an orientable totally geodesic hypersurface P" C M . Let ip : M" — > M be an 
orientable hypersurface with boundary E = t/j^dM), and let v stands for the outward 
pointing unit conormal vector field along dM C M . Then, along the boundary dM 
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and for every 1 < r < n, it holds that 

(6.1) Si = -si{^,iy) + {Au,iy), 

n-l 

(6.2) 52 = S2(e,^^)'~5l(e,^^)(A^.,^.)-^(A^.,e,)^ 

i=l 

(6.3) Sr = {-lYSr{Ll^y + i-lY-\sr-l{^,l^y-\Aiy,iy) 

n-l 

-{-iy-^{^, i^y-^ J2 sr-2in){A,., e,)^ 

i=l 

for 3 < r < n, and 

(6.4) {Triy,iy) = {~irsr{^,iy)y 
where s„ = and for every 1 < r < n — 1 

Sr = Sr{Ti, . . . , r„_i) 

zs t/ie r-th elementary symmetric function o/ti, . . . ,t„-i, the principal curvatures 
ofScP with respect to the outward pointing unitary normal, and ^ is the unitary 
normal field of P C M . 

It is not difficult to see that 1)6. 4|l establishes a very strong relationship between 
the transversality of M with respect to P along the boundary dAI, and the ellipticity 
on M of the r-th Newton transformation T,., when r > 1 (recall that Tq = /). That 
relationship between transversality and ellipticity will be actually one of the keys of 
the proof of our symmetry results fTheorem l7.ll Theorem 110. II and Thcorcm lll.l|l . 
In fact, saying that M is not transverse to P along its boundary dAI means that 
there exists a point p G dM such that iy){p) = 0, which implies from l|6.4|l that 
(TrV, i^){p) = 0, r > 1. Therefore we can conclude that if the Newton transformation 
Tr is positive definite on M for some 1 < r < n — 1, then the hypersurface M is 
necessarily transverse to P along its boundary. 

Observe that in the case where S'„ does not vanish on M and n > 3, transversality 
easily follows from expression (|6.3|) . In fact, by H6.3I) we have along the boundary 
dM 

Sn = {-ir-'sn-i{^,i^r-'{A,y,,,) 

n-l 

+ (-l)"-i(e, i^r-' J2 'n-2{n){A,^, ey\ 

i=l 

In particular, if there exists a point p G dM where (^, i'){p) ~ 0, then Snip) — 
(since n > 3). In the same way, if we assume that n > 2 and 5*2 is positive 
everywhere on M, then ()6.2(l also implies that M is transverse to P along the 
boundary. 

We summarize what was made above as follows. 
Proposition 6.2. Let E be an orientable (n—l)- dimensional compact submanifold 

n+l 71+1 

in an orientable totally geodesic hypersurface P" C M and let : Af" — > M 
be an orientable hypersurface with boundary S = ipidM). Then each one of the fol- 
lowing hypothesis individually implies that M is transverse to P along the boundary 
dM: 

• For a given 1 < r < n — 1, the Newton transformation Tr is definite positive 
on M. 
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• n > 3 and Sn ^ on M . 

• 52 > on M. 

7. Symmetry for hypersurfaces in Euclidean space 

The totally umbilic hypersurfaces of Euclidean space E"+^ are the totally geo- 
desic hyperplanes and the round n-spheres. They trivially have constant r-mean 
curvature for each r — 0, . . . ,n. Actually, the hyperplanes have vanishing r-mean 
curvature Hr = 0, and, after an appropriate choice of the unit normal vector field, 
the round n-spheres of radius g > have constant r-mean curvature Hr = 
Let us fix a hyperplane 11 C K"+^ and an (n — l)-sphere E C 11. Then the hy- 
perplanar round ball bounded by E in 11, and the spherical caps bounded by S 
(of radii greater than or equal to the radius of E) are examples of compact hyper- 
surfaces embedded into R"+^ with constant r-mean curvature and bounded by E. 
In this context, it was conjectured in |^ that these examples are the only com- 
pact embedded hypersurfaces in ]R"+^ with constant mean curvature and spherical 
boundary. Related to this conjecture we have the following symmetry theorem for 
hypersurfaces in Euclidean space f3J. 

Theorem 7.1. Let E be an strietly convex compact (n — I) -dimensional subman- 
ifold in a hyperplane 11 C M"^"'^, and let ip : M" — > R"+^ be a compact embedded 
hypersurface with boundary E. Let us assume that for a given 2 < r < n, the 
r-mean curvature Hr of M is a nonzero constant . Then M has all the symmetries 
of Ti. In particular, if the boundary E «s a round [n - l)-sphere o/R"+S then M 
is a spherical cap. 

Proof. It is not difficult to see that under the hypothesis above there exists at least 
one interior elliptic point of M, that is, an interior point of M where, after an 
appropriate orientation of M, all the principal curvatures are positive. In fact, 
since AI is not part of a hyperplane (because of Hr ^ 0), then one easily finds a 
radius R > and a point a G K^'^^ such that the closed round ball B{a, R) contains 
M and such that there is a point po € int(M) n dB{a, R) (englobe M with spheres 
of large radius until such a sphere touches M on one side at an interior point). In 
particular, in the chosen orientation the constant Hr = Hr{po) > is positive. The 
existence of an elliptic point, jointly with the fact that Hr is a positive constant, 
allows us to conclude that the Newton transformation T^^i is positive definite on 
M (see Proposition 3.2] and [21 p. 232]). Therefore, from Proposition lO it 
follows that M is transverse to 11 along the boundary dM. Our result then is a 
consequence of Theorem 7.3 in [23|. □ 

As a consequence of Theorem 17.11 we can conclude that the conjecture of the 
spherical cap is true for the case of embedded hypersurfaces with constant r- 
mean curvature in R"+^, when r > 2 T. 

Corollary 7.2. The only compact embedded hypersurfaces in K"+^ with constant 
r-mean curvature Hr (with 2 < r < n) and spherical boundary are the hyperplanar 
round balls (with Hr = and the spherical caps (with Hr a nonzero constant). 

Indeed, if M is not a hyperplanar round ball, then the constant r-mean curvature 
must be necessarily nonzero because there exists at least one interior elliptic point 
of M. In particular, when r = 2 saying that H2 is constant is equivalent to saying 
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that the scalar curvature is constant (see equation H2.7|l V so that the result reads 
as follows. 

Corollary 7.3. The only compact embedded hypersurfaces in R"+^ with constant 
scalar curvature and spherical boundary are the hyperplanar round balls (with zero 
scalar curvature) and the spherical caps (with positive constant scalar curvature). 

Our objective in Sections 1101 and 1111 is to extend the symmetry result given in 
Theorem 17. II to the case of hypersurfaces in hyperbolic space and hypersurfaces in 
sphere, as well as the corresponding solution to the spherical cap conjecture for 
the case of constant r-mean curvature, r > 2. A result of this type was first given 
by Nelli and Rosenberg in |19l Theorem 3.1] for hypersurfaces with constant mean 
curvature in hyperbolic space. On the other hand, the corresponding result for the 
case of hypersurfaces with constant mean curvature in sphere has been recently 
given by Lira As observed by Nelli and Rosenberg, their result could be 

extended to the case of constant r-mean curvature as soon as a certain flux formula 
could be established. In the next section, we will derive such a flux formula. 

8. A FLUX FORMULA 

In this section we will derive a general flux formula for the geometric configura- 
tion considered in Section^ in the case where the Riemannian ambient space M is 
equipped with a conformal vector field Y e X{M). Recall that the fact that Y is 
conformal means that the Lie derivative of the metric tensor of M with respect to 
Y satisfies 

for a certain smooth function G C°°{M). In other words, 

(8.1) ^vY,W) + {V.VwY) = 24>{V,W), 

for every vector fields V,W G X{M). 

In order to derive our general flux formula, let us consider Y^ G X{M) the vector 
field obtained on the hypersurface M by taking the tangential component of F, that 
is, Y^ = Y — /N, where / = (F, N). Most of the interesting and useful integral 
formulas in Riemannian geometry are obtained by computing the divergence of 
certain vector fields and applying the divergence theorem. The interesting integral 
formulas therefore correspond to vector fields with interesting divergences. Our idea 
here is to compute the divergence divjv/ {T^Y^). Using that VjjTr is self-adjoint 
for any tangent vector field U G X{M), an easy computation shows that 

n 

(8.2) divM {TrY^) = (divM Tr, Y)+Y, {V,Y^,Tre,), 

i=l 

where {ei, . . . , e„} is a local orthonormal frame on M and divA/ is given by 
in Lemma [3. II From the conformal equation (|8.1|l . we obtain 

2cj){TrU,U) = {Vt,^uY,U) + {VuY,TrU} 

= {^T^uY^, U) + /(VT„t/N, U) + {VuY^,TrU) + /(WN,T,[/) 
= {Vt^uY^.U) + {VuY^,TrU) - f{ATrU,U) - f{AU,TrU), 

that is 

(8.3) {Vt^uY^, U) + {VuY^,TrU) = 2(b{TrU, U) + 2f{ATrU, U) 
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Let us choose {ei, . . . , e„} a local ortlionormal frame on AI such that diagonalizcs 
A. We know then that it also diagonalizcs Tr with eigenvalues ^i,r, ■ • ■ ,^J'n,r, and 
therefore 

SO that from (|8.3|) we obtain 

Taking trace here and using H3.1|l and H3.2|l . equation H8.2|l becomes 

(8.4) divM (Trl"^) = (divM T,, y) + c,((/.H, + (y, N)i7,+i), 

where = (r + l)(^!^j^). Integrating now (|8.4|l on Af, the Stokes theorem implies 
the following integral formula for every < r < ?^ — 1, 

* {TrV,Y)ds = / diVA/ {TrY^)dM 
JdM JM 

(8.5) = / (diVM Tr,Y)dM + Cr I {<f)Hr + {Y,'H)Hr+l)dM 

JM JM 

Here dM denotes the n-dimensional volume element of M with respect to the 
induced metric and the chosen orientation, and ds is the {n — l)-dimensional volume 
element induced on dM . 

On the other hand, let Z?" be a compact orientable hypersurface in M with 
smooth boundary that satisfies dD = dM, such that M U I? is an oriented n-cycle 
of Af , with D oriented by the unit normal field no- We suppose that M U D = 
dQ, where is a compact oriented domain immersed in Af . From the conformal 
equation H8.1|l . we easily see that divjg- Y = {n + l)(/i. Therefore, from Stokes 
theorem we obtain that 

(8.6) [ (y, N)dM ^ - I (y, nD)dD + {n + I) [ cjydM, 
JM JD Jn 

where dD stands for the n-dimensional volume element of D with respect to the 

orientation given hy nu, and dM denotes the (n + l)-dimensional volume element 

on M. Now, from H8.5|l and 18.6|l we conclude the following general flux formula. 

n+l 

Proposition 8.1. Let i/' : Af" — > Af be an immersed compact orientable hyper- 
surface with boundary dM , and let f?" be a compact orientable hypersurface with 
boundary dD = dM . Assume that Af U D is an oriented n-cycle of M , and let 
N and no be the unit normal fields which orient Af and D, respectively. If the 
r-mean curvature Hr is constant, I < r < n, then for every conformal vector field 

Y G A'(Af ) the following formula holds 

(f {Tr-iv,Y)ds = / (divA/ r^-i,y)dAf + r( ) j cfiHr-idM 

JdM JM JM 

(8.7) ~r(^^HrJ {Y,nD)dD + {n + l)r(^^Hr J 4>dM , 

V is the outward pointing conormal to Af along dM . 

Formula H8.7|l becomes specially simple when the ambient space Af has constant 
sectional curvature, and the field y is a Killing vector field, that is, cj) — 0. In 
that case, the Newton transformations are divergence-free fCorollarv l3.2(l and from 
formula 1)8. 7|l we derive the balancing formula given by Rosenberg in |i24i Theorem 
7.2] (see also fHUZUHlEl for the case of constant mean curvature). 
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Corollary 8.2. If M has constant sectional curvature, then for every Killing vector 
field Y e X{M) the flux formula becomes 

(8.8) / {Tr-iv.Y)ds^ -r\]Hr ( {Y,nD)dD, 

JdM \rj Jd 

where v is the outward pointing conormal to M along dM. 

On the other hand, when the ambient space M has constant sectional curvature, 
and the field 1" is a homothctic (and non-Killing) vector field, then we may assume 
without loss of generality that 0=1 and H8.7|l becomes 



/ {Tr-ii^,Y)ds = -r(^]Hr [ {Y.nD)dD 
JdM \rj Jd 



^ Hr-idM + [n + l)r{^^Hr 



(8.9) +r / Hr-idM + {n+l)r\ i?,.vol(f7). 



As a consequence of (|8.9|) we obtain the following flux formula for r-minimal hy- 
persurfaces. 

Proposition 8.3. Let 4' '■ — > M ^ he a compact orientable hypersurface with 
boundary dM immersed into a Riemannian space of constant sectional curvature. 
If M is r-minimal in M , that is, = 0, then for every homothetic (non-Killing) 
vector field Y £ X(M) the following formula holds 

(8.10) / {Tr-ii^,Y)ds = r(^] I Hr-idM. 

JdM \rj Jm 

In particular, for minimal hypersurfaces in Euclidean space with boundary in a 
round sphere we have the following consequence. 

Corollary 8.4. Let S be an orientable (n — l)- dimensional compact submanifold in 
a round sphere S"{g) C M"^'^ of radius g, and let ip : M" — > R"+-'^ be an immersed 
orientable compact minimal hypersurface with boundary S ~ ip{dM) C S"'(g). Then 

vol(A/) < -vol(aM), 
n 

and equality holds if and only if M is orthogonal to S'^{g) along the boundary dM. 

Proof. Consider the radial vector field Y{p) — p in R"+^, which is a homothctic 
vector field in R"+^ with (/) — 1, and let ^ be the unit vector normal to §"(^?). Then, 
along §"(p) we have Y — g£^ and 18.10|l gives 



dM 

m 



nvol(M) = * {ly, g^)ds < g f ds ^ gvol{dM). 

JdM JdM 



Besides, equality holds if and only if ^ = along the boundary dM, or equivalently 
(see (EH)) (N, 0=0 along dM. □ 

Let us consider now the case of a hypersurface immersed into the hyperbolic 
space H""'"-'^. In that case, it will be appropriate to use the Minkowski space model 
of hyperbolic space. Write R"^^ for R"+^ with the Lorentzian metric 

= -dxl + dxl + h rfa;^+i. 

Then 

H"+i = {a; e R"+2 : (x,a;)^ = -1, xq > 0} 
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is a complete spacelike hypersurface in M"^^ with constant sectional curvature —1 
which provides the Minkowski space model for the hyperbolic space. 

Let S C 13"+^ be an orientable (n — l)-dimensional compact submanifold in a 
geodesic sphere S{a, g) of 11"+^ of center a S ]HI"+^ and geodesic radius g, and let 
ip : M" H"+^ be an orientable compact hypersurface with boundary S = ip{dM). 

Consider the vector field in IHI"+^ represented in this model as Y{p) = —a— {a, p)p 
for every p G H"^^. Observe that F is a conformal vector field in ]HI"+^ which is 
orthogonal to the geodesic spheres centered at the point a, with (j){p) = —{a,p) = 
cosh {g{p)) and \Y{p)\ = sinh {g{p)), where g{p) — distance(p, a) for everyp e H"^-^. 
Therefore, along S{a, g) we have Y — smhg^. Assume now that M is minimal in 
H"+i. Then, it follows from ijHiZIl that 



{i/,Y)ds — sinh g (p {i/,$^)ds — n / cosh{g)dM. 
dM JdM Jm 

Thus, since cosh (g) > 1, we conclude from here that 

nvol(Af) <n cosh{g)dM = sinh (p {h',^)ds < sinh £ivol(9M). 
Jm JdM 

Summing up, we have obtained the following result. 

Corollary 8.5. Let S be an orientable {n — 1)- dimensional compact submanifold 
in a geodesic sphere S{a, g) o/H"+^ of center a € H"+^ and geodesic radius g, and 
let ip : A{f" H""'""'^ be an immersed orientable compact minimal hypersurface with 
boundary E — 'ip{dM) C S{a, g). Then 

Yol(M) < ^^^vol(5Af). 

n 

Finally, let us consider the case of a hypersurface immersed into the sphere S""*"^, 

§«+i {a; = (a;o, . . . , Xn+i) £ M"+2 : {x, x) = 1}. 

Let S be an orientable (n— l)-dimensional compact submanifold in a geodesic sphere 
S(a, g) of §"+1 of center a e §"+i and radius g < tt/2, and let V : be 
an orientable compact hypersurface with boundary S = ipldM) C S{a, g). 

In this case, consider the vector field in S"+^ given by Y{p) = —a + {a,p)p 
for every p € S"+^, with singularities at the focal points {a, —a}. Observe that 
y is a conformal vector field in S"+^ which is orthogonal to the geodesic spheres 
centered at the point a, with = {a,p) = cos{g{p)) and = sin(^(p)), 

where g{p) = distance(p, a) for every p e §"+^. Therefore, along S{a, g) we have 
Y = sin £1^. Assume now that M is minimal in Then, it follows from H8.7|l 

that 



{iy,Y)ds — sin g (p {i/,£_)ds — n / cos {g)dM. 

dM JdM Jm 

Let us assume now that M is contained in the open hemisphere centered at a. In 
that case, it is clear that minjvf cos (g) = cos go, where go = maxdist(a, M), so that 

ncospovol(M) <n cos {g)dM = sing <p {h',S^)ds < sin gvol{dM). 
Jm JdM 

This leads to the following result. 
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Corollary 8.6. Let S be an orientable {n — 1)- dimensional compact submanifold 
in a geodesic sphere S{a, g) of S"+^ of center a G H"+"'^ and geodesic radius g, 
and let "ip ■ — > §"+^ be an immersed orientable compact minimal hypersurface 
with boundary S — ijj{dM) C S{a,g). Assume that M is contained in the open 
hemisphere centered at a. Then 

sin p ,„ , 

vol(M) < —vol(dM), 

n cos go 

where go = maxdist(a, M). 

9. Estimating the t-mean curvature by the geometry of the boundary 

In this section, we will describe an interesting application of our flux formula (|8.8() 
and the formula (|6.4|l . Let us consider the geometric configuration given in Propo- 
sition|^21 that is, let E be an orientable (n — 1) -dimensional compact submanifold 
in an orientable totally geodesic hypersurface P" C M, and let V' ■ -M" M""*" 
be an orientable compact connected hypersurface with boundary S = ilj{dM) and 
constant r-mean curvature Hr- Our objective here is to estimate Hr by the geom- 
etry of the boundary. Assume that there exists a Killing vector field Y G X{M) 
which is orthogonal to P. Then, we can write Y along the boundary dM both as 
Y = {Y,£,}^ and also as F = (Y, v)v + {Y, N)N, and using we obtain 

{Tr-iv,Y) = {Y,v){Tr-iv,v) = {-ir-\r-i{Y,£,){^,vY 

along the boundary dM . 

Let us consider D C P the domain in P bounded by S, and let us orient D by 
the unit normal field hd, so that MU D is an oriented n-cycle in M. Let us denote 
by hj the j-th mean curvature of S C P with respect to the outward pointing 
unitary normal rj, that is, 

. \ hj = Sj = Sj(ti, . . . , Tn-l), <j <n-l. 

In the case where the ambient space M has constant sectional curvature, then our 
flux formula (|8.8|) allows us to write 

(9.1) nHr [ {Y,nD)dD = i-iy <f hr-i{Y,0{^,i^Vds. 



Let us first apply formula H9.1|l to the Euclidean case, M 



pn.+l 



Theorem 9.1. Let E be an orientable (n — I) -dimensional compact submanifold 
in a hyperplane P C M""'""'^, and let ip : M" — > ]R"+-'^ be an orientable immersed 
compact (connected) hypersurface with boundary E — 'ip(dM) and constant r-mean 
curvature LI,., 1 < r < n. Then 

(9.2) < \Hr\ < (f \hr-l\ds, 

where h,-! stands for the (r — l)-mean curvature ofEdP, and D is the domain 
in P bounded by Y,. In particular, when E is a round {n — l)-sphere of radius g it 
follows that 

(9.3) 0<\Hr\<-^. 
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This estimate was first obtained in the case of constant mean curvature (r = 1) 
by Barbosa in 0]. 

Proof. Let ^ be the unit vector normal to P. Then ^ is a constant vector field in 
R""*"^, and therefore F = ^ is a Killing field in M"+^. On the other hand we also 
have that n^i = so that from H9.1|) we obtain 



dM 



< d) \hr-i\ds, 

dM 



n\Hr\yol{D) - 
which yields H9.2|l . 

In particular, when S = S"^^(p) is a round sphere of radius g, then we have 
that Ti — —1/g for every i = 1, . . . , n — 1, so that /i^-i — {—^Y^'^/0^~^- Besides, 
the domain D is an n-dimensional round ball of radius g, with volume n vo\{D) = 
g vol(§"~^ (£<)), and the estimate (|9.2|l becomes (jOJ- □ 

Let us consider now the case of a hypersurface immersed into the hyperbolic 
space H""*"^. As in Section |H1 it will be appropriate to use the Minkowski space 
model of H"+i, 

H"+i ={x= {xo, a^„+i) £ M'i+' : {x, x)^ = -1, xq > 0}. 

We may assume, up to an isometry of ]HI"+^, that the totally geodesic hyperplane 
P containing S is given by 

pn ^ ]jj„+l ^ ^ jg«+2 . ^^^^ ^ Q|_ 

In this case, the unit vector normal to P in H"+^ is given by ^{p) — e„+i = 
(0, . . . , 0, 1) G R"^^ for every p £ P. Observe that, for every arbitrary fixed point 
a £ P, the vector field given by 

Y{p) = -{p, a}e„+i + {p, e„+i)a, p e IHI"+\ 

is a Killing vector field on 11"+^ which is orthogonal to P, since at every p G P 

Yip) = -(p,a)e„+i = cosh {g{p))^{p), 

where g{p) is the geodesic distance along P between a and p. Let D be the compact 
domain D bounded by S in P, then njj = ±^ and from (|9.1|) we obtain 



hr^i cosh g{^, vYds 



(9.4) n\Hr\ / coshg dD = 

J D JT, 

Choose a G int(-D). Then min/jcosh^ = cosh^(a) — 1, so that from 19. 4|) we 
conclude that 

n\Hr\vo\{D) < n\Hr\ / cosh ^ dZ? < (p \hr-i\ cosh gds 

Jd JdM 



(9.5) < maxcosh^® \hr^i\ds. 

^ JdM 

In particular, when E is a geodesic sphere in P of geodesic radius g and a is chosen 
to be the geodesic center of S, then g{p) = g at every p £ E, |/ir-i| = coth'^~^(£i), 
and H9.5|l simply becomes 

(9.6) n\Hr\vo\{D) < coshgcoth''^"^(£i)vol(S). 

Moreover, in this case D is the geodesic ball in P of radius g centered at a, that is, 
D = {p e P : 1 < -{p,a) < coshg)}. 
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and 

Y. = dD = {p e P : -{p,a) = coshg}. 

Observe then that E is in fact a round (n — l)-sphere of EucHdean radius sinh g, and 
D is a round n-dimensional ball of Euclidean radius sinh g. Therefore, vol(S) = 
{n / sinh q)voI{D) and 1)9. 6|l simplifies to 

\Hr\< cotli'ig) 

We summarize this as follows. 

Theorem 9.2. Let E be an orientable {n — I) -dimensional compact submanifold 
contained in a totally geodesic hyperplane P C H"^^, and let ip : A/" —^ H""*"^ be 
an orientable immersed compact connected hypersurface with boundary E — ip{dM) 
and constant r-mean curvature Hr, 1 < r < n. Then 

0<\H.r\< \hr-l\ds. 

n vol(L>) JgM 

Here hr^i stands for the (r — \)-mean curvature of T. d P, D is the domain in P 
bounded by E, and C — max^ cosh p > 1, where g{p) is the geodesic distance along 
P between a fixed arbitrary point a £ int(_D) and p. In particular, when T, is a 
geodesic sphere in P of geodesic radius g, it follows that 

0<\Hr\< COth''£». 

Finally, let us consider the case of a hypersurface immersed into the sphere S"+^, 

S"+i ={x^ (xo, Xn+i) e M"+2 : {x, x) = 1}. 

We may assume, up to an isometry of §"+^, that the totally geodesic n-sphere P 
containing E is given by 

F" = S"+i n {x G M"+^ : Xn+i = 0}. 

In this case, the unit vector normal to P in §"+^ is given by ^(p) = e„+i = 
(0, . . . , 0, 1) G ]R"+^ for every p £ P. Observe that, for every arbitrary fixed point 
a £ P, the vector field given by 

Y{p) = (p, a)e„+i - {p, e„+i)a, p £ S"+\ 

is a Killing vector field on §"+^ which is orthogonal to P, since restricted to p € P 

Y{p) = (p,a)e„+i cos {g{p))^{p), 

where g{p) is the geodesic distance along P between a and p. Suppose that E is 
contained in an open hemisphere P+ of P determined by an equator S of P, and 
let D be the compact domain D bounded by E in P+. Then nu = and from 
(|9.1|l we obtain 



(9.7) n\H,. 



cos g dD 

D 



hr-icos g{$_,i') ds 

s 



Choose a £ int(_D). Since we are assuming that E = dD is contained in the open 
hemisphere P+, then < g < 7t/2 on D and min^ cos ^ > 0, so that from 1)9. 7|l we 
conclude that 



n 



Hr\ min cos gvol{D) < n\Hr\ / cosgdD< (p \hr-i\cosgds 

^ JD JdM 



(9.8) < maxcos^® \hr-i\ds. 

^ JdM 
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In particular, when E is a geodesic sphere in P of geodesic radius g < tt/2 and a 
is chosen to be the geodesic center of S, then g{p) ^ g at every p e E, \hr-i\ = 
cot^^^{g). Now a computation similar to that in hyperbolic space leads us from 
(liOll to 

\Hr\ < coi''{g), 

because, in this case S is in fact a round {n — l)-sphere of Euclidean radius sing, 
and D is a round n-dimensional ball of Euclidean radius sin g. Summing up, we 
can state the following result. 

Theorem 9.3. Let E be an orientable {n — 1)- dimensional compact submanifold 
contained in an open totally geodesic hemisphere P+ C §"+^ , and let ip : M" §"+^ 
be an orientable immersed compact connected hypersurface with boundary E = 
ip(dM) and constant r-mean curvature Hr, 1 < r < n. Then 

< l-ff.l < ^Jtttt / \hr-l\ds. 

n Yo\{D) Jqm 

Here h^-i stands for the (r — l)-mean curvature ofT,GP,Dis the domain in P-^ 
bounded by E, and C — maxs cos g/ min/^ cos g, where g(p) is the geodesic distance 
along P+ between a fixed arbitrary point a G int(Z3) and p. In particular, when S 
is a geodesic sphere in P+ of geodesic radius g < tt/2, it follows that 

0<\Hr\< C0f(?. 

10. Symmetry for hypersurfaces in hyperbolic space 

Hyperbolic space is rich in totally umbilic hypersurfaces. Besides the totally 
geodesic hyperplanes, there are the horospheres, the hyperspheres and the equidis- 
tant hypersurfaces. In all of them, the second fundamental form is proportional 
to the metric by a constant factor, and therefore they all have constant r-mean 
curvature, for 1 < r < n. After an appropriate choice of the unit normal vector 
field, hyperspheres have r-mean curvature bigger than 1, horospheres have r-mean 
curvature 1, and equidistant hypersurfaces have r-mean curvature in the interval 
(0,1). 

Let us fix a totally geodesic hyperplane P" C H"+^ and a geodesic sphere E" ^ C 
P" in ]HI"+-'^. Then each of the totally umbilic hypersurfaces above contains at least a 
compact domain M" with boundary being the sphere E. Those examples are called 
the spherical caps in hyperbolic space. That terminology is due to the fact that, 
working in the half-space model of hyperbolic space, after an appropriate isometry 
of H""*"^, the totally umbilic hypersurfaces above are given as intersections of ]HI"+^ 
with Euclidean spheres in M^^^. Because of the existence of these examples in 
13"+^, it is natural to consider the conjecture of the spherical cap in hyperbolic 
space. 

In this context, the corresponding result analogous to our Theorem 17.11 for the 
case of hypersurfaces in hyperbolic space can be stated as follows. 

Theorem 10.1. Let E"~^ be an strictly convex compact {n — 1)- dimensional (con- 
nected) submanifold of a totally geodesic hyperplane P" C ]HI"+^, and let M" C 
H"^"'^ be a compact (connected) embedded hypersurface with boundary E. Let us 
assume that for a given 2 < r < n, the r-mean curvature Hr of M is a nonzero 
constant. Then M has all the symmetries ofT,. In particular, when the boundary 
E is a geodesic sphere in P" C IHI"+^, then M is a spherical cap. 
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As a consequence of Theoreni llO.ll we can conclude, as in the Euclidean case, that 
the conjecture of the spherical cap is true for the case of embedded hypersurfaces 
with constant r-mean curvature in hyperbolic space, when r > 2. 

Corollary 10.2. The only compact embedded hypersurfaces in H"'"'"^ with constant 
r-mean curvature Hr (with 2 < r < n) and spherical boundary are 

• the geodesic balls of a totally geodesic hyperplane (with Hr = 0); 

• the geodesic balls of an equidistant hypersurface (with < \Hr \ < 1); 

• the geodesic balls of a horosphere (with \Hr \ — I); 

• the geodesic balls of a hypersphere (with \Hr \ > Ij- 

Proof of Theorem MU.lX Let us work in the half-space model of hyperbolic space. 
We may assume, up to an isometry of H"+^, that the totally geodesic hyperplane 
P is given by 

(10.1) P = {X = [Xi, . . . ,Xn+l) €W+^;\x\ = 1, Xr,+ l >0}. 

Let B be the connected component of IHI"+^\P containing the point (0, . . . , 0, 2) £ 
jjn+i^ We will first see that there exists an interior elliptic point, that is, a point 
Po G int(M) where all the principal curvatures of M are positive (after an appro- 
priate orientation of M). In fact, since Hj. is a nonzero constant, M cannot be 
enterily contained in P. After an inversion with center (0, . . . , 0) £ ]R"+^ which 
fixes P (an isometry of H""'""'^), if necessary, we may assume that M r\B ^ Let 
C C P the geodesic sphere in P given as the boundary of a geodesic ball in P 
centered at the point (0, . . . , 0, 1) and containing S. Let us consider C 11"+^ the 
equidistant sphere with center on the vertical geodesic through the center of C such 
that n P = C, and such that the exterior angle between and the asymptotic 
boundary of H"+-^ is f — £ > 0. Since F^ P as £ — *■ 0, and taking into account 
that M B ^ we may choose £ > such that F*^ n M ^ 0. Besides, since S is 
contained in the geodesic ball in P bounded by C, then the points in F*^ nM are in- 
terior points of M. Now, for every i > 0, let us consider Ff C H""*"^ the equidistant 
sphere in IHI"+^ obtained from F^ by an homothety centered at (0, . . . , 0) G M"+^ 
(which is also an isometry of 11"+^), and let us define Fq = F*^. If i is large enough, 
then Ff englobes M\ thus, we may find tg > such that M is tangent to at a 
point poi which is necessarily an interior point of M . Finally, it is easy to conclude 
that the normal curvatures of M at poi with respect to the normal direction of 
the mean curvature vector of F^^ , are greater or equal to those of Ff^ , which are 
positive. In particular, choosing the appropriate orientation of M, all the principal 
curvatures of M at po are positive. 

Therefore, we may assume that — Hr{po) is a positive constant. This implies 
that for every 1 < j < r — 1, the Newton transformation Tj is positive definite on 
M (see |H1 Proposition 3.2]), and in particular the mean curvature is positive on 
M, so that we may assume that M is oriented by the mean curvature vector field. 
From Proposition 16 . 21 we know that M is transverse to P along the boundary dM. 
This implies that, in a neighborhood of the boundary dM , M is contained in one 
of the two connected components of ]HI"+^\P, which, without loss of generality, can 
be assumed to be B. Beside, we may also assume that M is globally transverse to 
P. 

In this situation, we will prove that M is above P, that is, M C B. Let us con- 
sider M the connected component of M n B containing E. Then, M is a compact 
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embedded hypersurface in H"+^ with boundary dM contained in P. If the bound- 
ary dM were connected, then M — M and there is nothing to prove. Our objective 
is to show that actuaUy dM must be connected. We will prove it by showing that 
assuming that dM is not connected yields a contradiction. 

Thus, let us assume that the boundary dM consists of a finite number of disjoint 
connected compact emdedded (n— l)-dimensional submanifolds C P (0 < i < fc), 
with Eq = S. We orient this configuration as in Section^ with M oriented by the 
mean curvature vector of M. Let v be the outward pointing conormal to M along 
each connected component of dM. Then, the mean curvature vector of M , together 
with v, allows us to orient each E^. Let rj be the unitary vector field normal to E in 
P which points outward with respect to the domain D bounded by E in P, and let 
^ be the unique unitary vector field normal to P which is compatible with rj and 
with the orientation of E. Now, there exists a unique choice for the unitary vector 
field r]i normal to E^ in P which is compatible with the orientation of E^ and with 
the orientation of P given by ^. We remark that we cannot ensure here that, for 
i ^ Vi points outward to the domain Di bounded by E^ in P. In this way, we 
have that formula (|6.2(l holds at each point p G dM with r ~ 1, giving 

(10.2) {T,iy,iy){p)^~si{p){t.iy){p) 

Here si denotes the trace of the shape operator, with respect to rji, of the inclusion 
Ei C P which contains the point p. 

As E is a compact strictly convex submanifold of P and 77 points outward of D, 
then si < on E. On the other hand, as Ti is positive definite on M, it follows 
from ((Tir^ that ly) > on dM. Besides M C B implies that (C, > on 
each component of dM. Hence, along E, the mean curvature vector of M points 
into D. Therefore, if dM has a connected component contained in the interior 
of D, then there exists at least one component E^, for some i > 1, contained in 
the interior of D on which the mean curvature vector of M points outward to the 
domain Di C P bounded by E^ in P. As {£,,v) > on E.^, then rji must point 
into Di. This contradicts the formula H1U.2|I . because if rji points into Di, then we 
can easily conclude from the compactness of E^ that there must be a point p e E.^ 
where si{p) > 0. It then follows that the connected components of dM must be all 
contained in P\D. 

Now, let us assume that there exists one of them, say Ej {j > 1), which is 
homotopic to E in P\D. Without loss of generality, we may assume that, between 
Ej and E there is no other component of dM which is homotopic to E in P\D. 
We showed above that, along E, the mean curvature vector of M points into D. 
Therefore, along Ej , the mean curvature vector of M must point outward of the 
domain Dj C P bounded by Ej in P. Since (^, > on Ej, it then follows that 
the unitary vector field rjj normal to Ej in P points into Dj. This situation gives 
again a contradiction with formula (|10.2(l . because if r]j points into Dj, then there 
must be a point p e Ej where si{p) > 0. 

Finally, it only rests the case where dM has a connected component E; (/ > 1) 
which is contained in P\D and is null homotpic in P\D. However, this final 
possibility is discarded by using the Alexandrov reflection technique |Ij , exactly as 
in the proof of [HI Theorem 1] or |19l Theorem 3.1]. For the sake of completeness, 
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we will include here the argument. Let 7 an infinite length geodesic in P starting 
at a point of D and intersecting S; in at least two points. 

Consider a family Q{t), t < 00, of geodesic hyperplanes of 11"+^ orthogonal 
to 7, such that for each g G 7 there exists exactly one Q{t) which intersects 7 
orthogonally at q. Each Q{t) is orthogonal to P, so a hyperbolic simmetry through 
Q{t) leaves P and B invariant. Now we apply Alexandrov reflection method to M 
(observe that this can be done because the equation Hr = constant > 0, under 
the existence of an elliptic point, is an elliptic equation ^3]). For t large enough 
Q{t) is disjoint from M. As t decreases, there must exist a first point of contact 
of some Q{t) with M. One continues to decrease t and considers the symmetries 
of M through the geodesic hyperplanes Q{t). Sinces 7 intersects S; in at least 
two points, there must exist some hyperplane Q(to) such that the symmetry of M 
through (3(to) will touch M at an interior point. This occurs at an interior point 
since E is convex and 7 intersects S exactly at one point. Thus, M is invariant 
under symmetry through Q{to), which is impossible (for M would then be part 
of an embedded closed manifold with constant r-mean curvature, hence, a sphere. 
But a sphere cannot meet P in more that one component). 

Summing up, we conclude from the reasoning above that dM has no other con- 
nected component on P except of E, and therefore M C B. Now that we know that 
M is above P and transverse to P along dM, the proof finishes applying again the 
Alexandrov reflection method to M U D, exactly at in the flnal step of the proof of 
Pl Theorem 2.1]. □ 

11. Symmetry for hypersurfaces in sphere 

The totally umbilic hypersurfaces of S"+^ are given by the intersections of 
§"+1 with the hyperplanes of Euclidean space M"+^. When the hyperplane passes 
through the origin of R"+^, they are totally geodesic, and when the hyperplane 
is an affine hyperplane, they are totally umbilic. We will refer to them as totally 
geodesic n-spheres and totally umbilic n-spheres of respectively. They all 

have constant r-mean curvature. After an appropriate choice of the unit normal 
vector field, the totally umbilic n-spheres have r-mean curvature Hr = cot^{g), 
where ^? > denotes the geodesis radius of the convex geodesic ball of whose 
boundary is the totally umbilic n-sphere. 

Let us fix a totally geodesic n-sphere P" C S"+^ and a geodesic sphere S""^ C 
P" in §"+^. Then, fixed a value for Hr, there are two compact domains M" and 
M2 of a totally umbilic n-sphere of §"+^ whose boundaries are the geodesic sphere 
S. These examples are called the spherical caps in S""*"^. As in hyperbolic space, 
because of the existence of these examples in S"+^, it is also natural to consider the 
conjecture of the spherical cap in §"+^. In this context, the corresponding result 
for the case of hypersurfaces in S"+^ can be stated as follows. 

Theorem 11.1. Let E"^^ be a convex (n— 1)- dimensional submanifold of a totally 
geodesic n-sphere P" C and let M" C §"+^ be a compact (connected) embed- 

ded hypersurface with boundary E. Let us assume that M is contained in an open 
hemisphere and that the r-mean curvature Hr of M is a nonzero constant, 

for a given 2 < r < n. Suppose that the convex disc D bounded by Yi in P contains 
a focal point of Pi H P , where Pi = c^S""*"^. Then M has all the symmetries of E. 
In particular, when the boundary T, is a geodesic sphere in P" C S""*"^, then M is 
a spherical cap. 
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Corollary 11.2. Let M be a compact (connected) embedded hypersurface in §"^^ 
with constant r-mean curvature Hr ^ (with 2 < r < n) and spherical boundary 
contained in a totally geodesic n-sphere P" C S""*"^. Suppose that the convex disc 
D bounded by the spherical boundary of M in P contains a focal point of Pi H P, 
where Pi ~ Then M is a spherical cap. 

As we had pointed out before, corresponding results for r = 1 can be found in 

m 

Before go further, it is needed to fix a suitable notion of symmetry in the spherical 
space form. This is done in the definition below. 

Definition 11.3. We say that a totally geodesic n-sphere Q is a n-sphere of sym- 
metry of a subset S of S""*"^ if for each point p € S and any complete geodesic 7 
perpendicular to Q and containing p, we have p G 5, where p is the point of 7 such 
that p and p lie in opposite hemispheres of Q at distance less than or equal to ^ 
and dist(p, Q) = dist(p, Q). 

We observe that the choice of D in this section is compatible with the orientations 
established in the Section 4, what allows us to use the calculations made in the 
earlier parts of the article. 

Proof. Let a G §"+^ and consider Pi = {a; £ : {x, a) = 0} the totally geodesic 
n-sphere which defines the open hemisphere = {x G S"+^ : {x, a) > 0} where 
M is contained. Now, we may assume without loss of generality that the totally 
geodesic n-sphere containing the boundary of M is 

P = {xe §"+1 : {x, eo) = 0}, a ^ cq. 

Our first objective is to see that there exists an interior elliptic point of M, that 
is, a point po G int(Af) where all the principal curvatures of M have the same 
sign. To see it, let Bt{a) C S"^^ be the geodesic ball with center a and geodesic 
radius t, where < t < 7r/2, and let St{a) = dBt{a) be the corresponding geodesic 
sphere. Since M is compact and M C S""*"^, there exists a minimum value t' 
such that M C Bt'{a), and a contact point po G M n St{a). Observe that the 
height function {x, a) on M attains its minimum value precisely at that contact 
point. Therefore, if such a contact point is an interior point of Af , then it is also 
a tangency point and all the principal curvatures of A/, with respect to the unit 
normal vector field of St'{cL), are positive at po- If the contact point is a boundary 
point, then we can consider a geodesic ball Bt{a) with t > t' so that i3t(a) n S — 0. 
Now, we can simultaneously move the center a of the geodesic ball and decrease 
its radius, keeping always M contained in the interior of this geodesic ball, and 
we consider the intersection of this geodesic ball with §"^^. From this process it 
follows that either some geodesic ball Bt[a') fl §"^^ is tangent to M at an interior 
point, or M is entirely contained in the totally geodesic n-sphere P. However, 
the second possibility cannot happen because Hj. is a nonzero constant. Then, 
reasoning as above, such an interior tangency point is an elliptic point of M . Thus, 
we may always (including when r =even) assume that the r-mean curvature Hr of 
Af is a positive constant. This implies that Tj is positive definite on M , for each 
1 < i l£ f — \, and, since H = Hi > 0, we may orient M by the mean curvature 
vector. By the Proposition 16.21 we conclude that M is transversal to P along its 
boundary dM. So, there is a neighbourhood W of S in Af contained in only one of 
the hemispheres P+ and P~ determined by P. We fix C P+. 
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Let D C P he the domain bounded by S which does not contain points of 
Si := Pi n P. Denote by int(Z?) the interior of D in P and by ext{D) the subset 
P — D. According to this notation, we have 

Claim 11.1. IfAI int(D) ^ 0, then M n ey±{D) ^ 0. 

We omit the proof of this claim since it foUows the same guidehncs from the 
similar one for the hyperbolic case (see Section 10). 

For guarantee that Af n P = S it suffices then, by the Claim, to prove that 
Mnext(i5) = 0. Suppose otherwise, that is, suppose that Mnext(-D) ^ 0. So, we 
may assume, without loss of generality, that Mnext(D) consists of a finite number 
of disjoint connected embedded submanifolds of P. 

In order to apply correctly the reflection procedure, we consider now on M the 
connected component containing E of the topological embedded submanifold N 
obtained after excision of small annuli in M surrounding each one of the domains 
Di bounded by the components of E fl intD containing no points of E and gluing 
domains homeomorphic to each Di at the boundary of these annuli (see [H] for the 
similar device in R^). This construction allows us to consider M U D separating 
gn+i connected components. By 17 we denote the component that contains 

no points of Ei. Note that the set M U ext(D) is not diminished in this process. In 
fact, the only components of M U ext(D) that could be discarded are the ones that 
contains points in int(D) and points in ext(D), whose existence should oblige the 
mean curvature vector to point outside f2, contradicting the Maximum Principle 
applied to geodesic graphs in S"+-^ (see [TT]'l. 

Case 1. Suppose initially that there exist components Efc of M n ext(D) homolo- 
gous to zero in P — int(D). For each fc, denote by Mk the connected component of 
M which has boundary E^ and contains no points of E. We note that Mk contains 
points of P~ in a neighbourhood of E^^. 

We fix El = {x = (0, 0, X2, . . . , Xn+i) G §"+^} and a;i > throughout the 
hemisphere of P which contains no points of E. Define Pi(t), < t < tt, as 
the family of totally geodesic n-spheres such that Pi{t) H P = Ei, for all t, and 
Pi (a) = Pi, where a is the angle between Pi and P. The normal vector to Pi(t) 
is given by nt — (cos t, — sin t, . . . ,0,0). 

For t, < t < a, let M^{t) be the set {x e M; {x,na) > Oand {x,nt) < 0} and 
let Mfc(t) be the reflected image of M^{t) through Pi{t), i.e., Mfe(i) = {i e 
§"+1; 5 = x- 2{x, nt)nt, x £ M-{t)}. 

By the fact that Pi (a) H Mk — 0, there exists to, < to < a, such that 

(i) Pi(to)nMfc ^0; 

(ii) Pi(i) n Mk = 0, for aU t > to. 

So, Mk is tangent to Pi(io) at their common points and there is a neighbourhood 
of each one of these points in Mk which is a geodesic graph over a domain in Pi(io). 

Thus, unless to — 0, we have that Mk{t) C P~ for t sufficiently close to to. 
However, for to — 0, we consider a rotation of P by a small angle, fixing Ei, to 
return to the previous situation. 

We claim that Mk is a geodesic graph over the domain Dk in P bounded by 
T^k which contains no points of Ei, with Mk{Q) C int(r2). In particular, Mk is not 
perpendicular to P at points of E^^. Otherwise, there exist k and ti G [0,io) f^or 
which occurs one of the following possibilities: 

(i) Mk{ti) n M contains interior points of Mk; 
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(ii) Pi(ii) is perpendicular to Mk at points of dMk{ti); 

(iii) M/c(ii) n M contains points of {M - E) - M^.; 

(iv) Mk{ti) contains points of S. 

The cases (i) to (iii) are all ruled out by the Maximum Principle. In fact, 
otherwise, Pi(ti) should be a sphere of symmetry and M a compact hypersurface 
without boundary (see P- 572-573). 

Thus, we conclude that the points of Mfe(ti) away from S are contained in int(r2). 
Since there exists a neighbourhood W of S in M as above, the reflected image of 
Mk{ti) through P = P(7r) is not contained in fi, if we suppose Mk{ti) n S 7^ 0. In 
particular, the reflection of Mk{ti) through P is not contained in the open domain 
bounded by Mfe(ti) in int(r2). Therefore, a sphere Pi{t), a < t < tt, should exist 
such that the reflected image of Mfc(ti) through Pi(r) is tangent to Mfc(ti) and, in 
this way, Pi{t) is sphere of symmetry of Mfe(ti). Hence, since the portion of Mk{ti) 
lying between Pi (a) and Pi{t) does not contain points of dMk{ti) — dMk{ti) in 
Pi(ti), we obtain a contradction, proving that the case (iv) does not occur at any 
instant t E [0, io). 

Notice that it is equally impossible to have Mfe(ti) tangent to M at points of 
opposite orientation, because if it is the case, then the reflected image of a portion 
of Mfc would have left int(rj) before ti. 

So, we conclude from the impossibility of the cases (i) to (iv), for each t e [0, to), 
that Mk {t) is a geodesic graph over the domain in Pi (t) bounded by dAI^ (t) which 
does not contain points of Ei, with Mk{t) C int(r2), proving the claim. Besides this, 
we guarantee that Af^T (t) is not perpendicular to Pi (t) at any point of dM^ (t) . 
Case 2. Now, suppose there exist components of M n ext(P') homologous to E. 
We will prove that whenever exist such components, they are graphs over domains 
in P. 

This case is handled as in 17 , with minor modifications concerning the utiliza- 
tion of the flux formula there, which must be changed by the appropriate formula 

EH 

Now, as above, define for t G (a, ^] the submanifold of M given by M^(t) = 
{x G P^; {x,na) > Oand, (x, rij) < 0} and its reflected image through Pi{t) as 
M{t) = {i e = X - 2{x,nt)nt, x G M-{t)}. 

Since M fl Pi =0, either M is contained in the open hemisphere determined by 
Pi(-|) containing E, or exists to G («, f ] such that 

(i) Pi(to)nM^0; 

(ii) Pi{t) n M = 0, for all a < t < to- 

For to = -I, there is a neighbourhood of M that is a graph over a domain of 
A(f). 

If to < suppose that there exists ti G (to, for which holds one of the 
statements below: 

(i) M(ti) is tangent to M at interior points; 

(ii) Pi(ti) is perpendicular to M at some points of M n Pi(ti); 

(iii) M(ti) n E 7^ 0. 

If (i) or (ii) occurs, then Pi(ti) is a sphere of symmetry of M. However, E is 
contained in only one of the hemispheres determined by Ei = Pi(ti) n P on P. 
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Suppose (iii) occurs; if exists p S M~ (ti) such that p G S, then p and p are points 
at the same distance from Pi(ti) on a geodesic S perpendicular to Pi(ti). If ii = f , 
then p S P, since P is totally geodesic. If ti < ^, we have dist(p, Pi(ti)) < -1; 
thus, dist(p,p) < 2ti < TT what implies p € P~. Both situations contradict the fact 
that M-{ti) C P+. ^ 

We conclude that M(ti) C int(f7), for all t 6 (a, |]. Furthermore, M-(|) is a 
geodesic graph over the domain bounded by dM~{^) in Pi(f ) containing points 
of Q. 

Let p ^ D the geodesic center of Si and a an arc of geodesic starting from p 
passing through S and crossing orthogonally Si. We may assume, initially, that 
the component Sfc^ = dMk^ of M D ext(£>) nearest from Si in the direction given 
by (T is homologous to zero. Alodifying slightly the direction of a, if necessary, we 
may assume that a crosses Sfe(, at least twice. 

For each point (j{t), < t < d, we consider the intersection Q{t) of S"+^ and the 
Euclidean hypcrplane containing the origin of R"+^ and perpendicular to {xq = 0} 
whose normal vector is cr'(i). Denote by Q~{t) the hemisphere determined by Q{t) 
containing (j\t,d\ and by Qt the reflection through Q{t). 

As we have proved, the portion of M in the hemisphere Q~{d) determined by 
Q{d) = Pi(|), if it is not empty, is a geodesic graph over a domain in Pi(^) n P+ 
at distance less than ^ from the sphere Pi(f )• This remains true, for t sufficiently 
close to d. By the choice of S/j„. we have that the first point of contact, if exists, 
between the planes Q{t) and M n P~ must be in S^^. More precisely, there exists 
to € (0, d) such that we have: 

(i) MnQ~ (t) is contained in the portion of the geodesic cylinder over a domain 

of Q{t) contained in P+ and Q,{M n Q^it)) Cflf) P+, for all t < to; 

(ii) M n P^ n Q(to) is a non-empty subset of S/j^. 

These statements follow from the fact that Mkg is, as proved above, contained 
in the geodesic cylinder over a domain in P and Q{to) is a totally geodesic sphere 
perpendicular to P. So, since a crosses Sfc„ at least twice, and M^g is compact, 
there exists ti S {0,to] so that MriQ~{t) is contained in the geodesic cylinder over 
a domain of Q{t), in such a way that Qt{M n Q^{t)) C int(ri), whenever t > ti. 
Furthermore, one of the following assertions holds: 

(i) Qt^ (M n Q~ {ti)) is tangent to Mk^ at points not belonging to Q{t\) with 
the same orientation; 

(ii) Q{tx) is perpendicular to M{^Q~(t\) at points of Q{ti) or, equivalently, 
Qti (M n Q-{ti)) is tangent to M at points of Q{ti). 

In any case, Q{t\) should be a sphere of symmetry of M. Let p' be the last 

point of S in (7[0, d). The distance between p' and Q{ti) is loss than ti. Thus, 
prolonging cr until the point Qt^ {p'), we obtain an arc of geodesic of lenght strictly 
less than 2ti < n. Since Q{ti) is a sphere of symmetry of M and, in particular, of 
S, wc have that Q{p') is a point of S. However, since that S is convex, cr does not 
return to S until it has just crossed all of the hemisphere determined by Si in P 
which does not contain points of S, that is, just after t > tt. As 2ti < n, we have 
a contradiction. From this contradiction, wc conclude that there is no components 
of M n ext(£') homologous to zero outside the region in P — Si bounded by S and 
some component of M fi ext{D) homologous to S; otherwise, there exists at least a 
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direction u starting from -p so that the component of M n ext(I?) nearest from Ei 
in its direction is homologous to a constant. 

Now, suppose Efeo is homologous to S. By construction, it is clear that <j crosses 
each component Sj, of M n ext(_D) homologous to S at least once. So, proceeding 
as in |17| we find a sphere of symmetry of M before reach S, a contradiction. 

At the moment, we have proved that M is contained in P+ and that M n P = S. 
Furthermore, M~(|-), if it is not empty, is a geodesic graph over a domain in Pi(^) 
having height less than ^ and its reflected image through Pi (f-) is entirely contained 
in int(ri). 

Let then P be a sphere of symmetry of S and q E Rf) D. Let fi be the geodesic 
perpendicular to R starting from q and reaching a point g' G Si. We define R{t), 
< t < 2tt, as the intersection of §"+^ and the Euclidean hyperplane containing 
the origin of and perpendicular to {xq = 0}, whose normal is ^'(t)(o,...,o) • It is 
clear that P(0) = R. Suppose that R{d) and Pi(f ) coincide. Then, the facts above 
imply that we have no touching points until the time t = d on the reflection process 
through the spheres R{t). However, since M is compact and M D P — S, there 
exists ti G [0, d) such that R{ti) is a sphere of symmetry of M and, in particular, 
of S. Since R and R{ti) are both perpendicular to ^, it follows from the convexity 
of S that R — R{ti), i.e., that P is a sphere of symmetry of M. 

If P((i) and Pi ( -J ) are distinct spheres, let E2 = nP((i) and consider the 

totally geodesic spheres T{t), < t < ao, obtained by rotation, fixing E2, of Pi(^) 
towards P(d), with T(0) = P(f ) and T(q!o) = R{d). It is clear that E2 = ntT{t). 
Moreover, we have that T{t) n E = 0, for all t, since each T{t) is contained in the 
domain C of S"+^ bounded by Pi(-|) and R{d) that does not contain points of E. 
Denote T~ (t) and T(t) as before. 

By continuity, we have that, for t close enough to 0, each component of MDT^ (t), 
when this set is not empty, is still a geodesic graph over a domain T{t) at distance 
from T{t) strictly less than ^. Furthermore, since M is compact, it is possible to 
consider t sufficiently small so that T{t){M n T~{t)) C int(il). Thus, either this 
remains true for each t G (0, ao], or there exists ti G (0, ap] such that one of the 
following situations occurs: 

(i) {M n T^{ti)) is tangent to M at points not belonging to T{ti) with the 
same orientation; 

(ii) T{ti) is perpendicular to M C\T^{ti) at points of T{ti) or, equivalently, 
Tt^{M r\T-{ti)) is tangent to M at points of T{ti). 

In these cases, T{ti) should be a sphere of symmetry of M and, in particular, of 
E. However, this contradicts the fact that there are no points of E in C. Therefore, 
we conclude from this contradiction that 7^q(M n T~(ao)) is contained in and 
that M n T~(ao) is either empty or a graph over r(ao) = R{d). In this way, we 
return to the previous case. The theorem is proved. □ 
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